Replace tqdm with a logger for different modules. #3
Conversation
MPMPMPMPMPMPMP
force-pushed
the
logger
branch
2 times, most recently
from
f840587 to
f9c45ea
Compare
MPMPMPMPMPMPMP
force-pushed
the
logger
branch
from
c64defa to
829cde1
Compare
|
Hey, first thank you very much for the idea and implementation! Three things I spotted:
I will take some time tomorrow to walk through it in detail. |
This is the with all the loggers set to info if you set everything to warning except the optimization, then you see only the steps. |
|
I really like to use the progress bar where I can see the current energy without scrolling through a wall of text. |
JanLuca
left a comment
JanLuca
left a comment
There was a problem hiding this comment.
Besides the minor comments, I like the PR :) Thank you very much for the work!
Only the optimizer.py file I could not really review since the change of the indention makes the comparison hard
| from __future__ import annotations | ||
|
|
||
| import logging | ||
| from typing import Any | ||
|
|
||
| from varipeps import config as _cfg_mod # uses the global config instance | ||
|
|
||
| _LOGGING_INITIALIZED = False | ||
|
|
||
| def _to_py_log_level(level: Any) -> int: | ||
| # Accept both enum values and raw ints; OFF disables effectively | ||
| try: | ||
| val = int(level) | ||
| except Exception: | ||
| val = logging.INFO | ||
| if val == 0: # OFF | ||
| return logging.CRITICAL + 10 | ||
| return val | ||
|
|
||
| def init_logging(cfg: Any | None = None) -> None: | ||
| """ | ||
| Initialize logging based on the provided config (or global config). | ||
| Safe to call multiple times; replaces handlers to avoid duplicates. | ||
| """ | ||
| global _LOGGING_INITIALIZED | ||
| if cfg is None: | ||
| cfg = _cfg_mod.config | ||
|
|
||
| root = logging.getLogger("varipeps") | ||
| # Remove old handlers to prevent duplicate logs | ||
| for h in list(root.handlers): | ||
| root.removeHandler(h) | ||
|
|
||
| root.setLevel(_to_py_log_level(getattr(cfg, "log_level_global", logging.INFO))) | ||
| root.propagate = False | ||
|
|
||
| # fmt = logging.Formatter( | ||
| # fmt="%(asctime)s %(levelname)s %(name)s: %(message)s", | ||
| # datefmt="%H:%M:%S", | ||
| # ) | ||
|
|
||
| fmt = logging.Formatter( | ||
| fmt="%(asctime)s %(levelname)s %(message)s", | ||
| datefmt="%Y-%m-%d %H:%M:%S", | ||
| ) | ||
|
|
||
| if getattr(cfg, "log_to_console", True): | ||
| sh = logging.StreamHandler() | ||
| sh.setFormatter(fmt) | ||
| root.addHandler(sh) | ||
|
|
||
| if getattr(cfg, "log_to_file", False): | ||
| fh = logging.FileHandler(getattr(cfg, "log_file", "varipeps.log")) | ||
| fh.setFormatter(fmt) | ||
| root.addHandler(fh) | ||
|
|
||
| # Per-module levels | ||
| logging.getLogger("varipeps.optimizer").setLevel( | ||
| _to_py_log_level(getattr(cfg, "log_level_optimizer", logging.INFO)) | ||
| ) | ||
| logging.getLogger("varipeps.ctmrg").setLevel( | ||
| _to_py_log_level(getattr(cfg, "log_level_ctmrg", logging.INFO)) | ||
| ) | ||
| logging.getLogger("varipeps.line_search").setLevel( | ||
| _to_py_log_level(getattr(cfg, "log_level_line_search", logging.INFO)) | ||
| ) | ||
| logging.getLogger("varipeps.expectation").setLevel( | ||
| _to_py_log_level(getattr(cfg, "log_level_expectation", logging.INFO)) | ||
| ) | ||
|
|
||
| _LOGGING_INITIALIZED = True | ||
|
|
||
| def ensure_logging_configured(cfg: Any | None = None) -> None: | ||
| """ | ||
| Initialize logging once on first call; subsequent calls are no-ops. | ||
| """ | ||
| global _LOGGING_INITIALIZED | ||
| if not _LOGGING_INITIALIZED: | ||
| init_logging(cfg) |
There was a problem hiding this comment.
I would move that to the config file since there one could catch changes of the configuration and update the logger instances for the new values. Will prepare a commit to showcase that.
The HDF5 file stores config values like log_level_global as a numeric (e.g., numpy.int64). When loading, that integer is passed into VariPEPS_Config. In setattr, the field type for log_level_global is the Enum LogLevel, so integers must be coerced to LogLevel. Your version’s coercion path doesn’t catch your value, so it falls through and raises: Type mismatch for option 'log_level_global', got '<class numpy.int64>', expected '<enum 'LogLevel'>'. Why it falls through The loader passes a numpy integer (or a 0-d/1-d array) instead of a Python int. The Enum branch in setattr is too strict about the numeric checks, so it doesn’t convert that value into LogLevel.
MPMPMPMPMPMPMP
force-pushed
the
logger
branch
from
20cd0b0 to
848d748
Compare
2 participants
TLDR;
This pull request introduces a major refactor to the logging and progress reporting system across the codebase, replacing the use of
tqdm_loggableand custom print/debug statements with Python's standardloggingmodule. It also adds configurable logging levels and destinations to the configuration, and updates many routines to use structured logging for progress, warnings, and informational output.If you think something is missing please let me know!
Simple usage (just inlude this in your
It makes the log look the following way (depending on the log levels you choose):
(please don't tell me to remove the emojis I like the eye candy ... )
Key changes include:
Logging System Overhaul:
LogLevelenum and added multiple logging-related configuration options (e.g.,log_level_global,log_to_console,log_to_file) to theVariPEPS_Configdataclass, allowing fine-grained control over logging behavior. [1] [2]tqdm_loggableand associated progress bar logic from the codebase, including in__init__.py,line_search.py, andoptimizer.py. [1] [2] [3]debug_printandtqdm.write) with calls to the standardloggingmodule at appropriate levels (info,debug,warning) throughout the optimization and CTMRG routines. [1] [2] [3] [4]CTMRG and Optimization Routine Updates:
ctmrg/routine.pyto use a module-level logger, providing structured logs for convergence, step progress, and parameter changes, including timing information for key operations. [1] [2] [3] [4] [5] [6] [7] [8]optimization/line_search.pyandoptimization/optimizer.pyto use their own loggers, replacing progress bar and print statements with logging, and providing detailed info and warning logs for line search steps, autosaving, and convergence issues. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14]Configuration and Code Cleanup:
These changes collectively modernize the codebase's output and monitoring capabilities, making it easier for users and developers to control and interpret runtime information.
Logging System Overhaul:
LogLevelenum and new logging configuration options toVariPEPS_Config, allowing fine-grained control over logging levels and destinations. [1] [2]tqdm_loggableand associated progress bar logic, standardizing on Python'sloggingmodule. [1] [2] [3]CTMRG and Optimization Routine Updates:
ctmrg/routine.pyandoptimization/line_search.pyto use structured logging for step progress, convergence, and parameter changes, including detailed info and warning messages. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15]optimizer.pyto use logging for autosaving, convergence checks, and progress reporting, replacing all progress bar and print-based feedback. [1] [2] [3] [4] [5] [6] [7] [8]Configuration and Code Cleanup: